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UNIFORM MARKOV RENEWAL THEORY AND RUIN 
PROBABILITIES IN MARKOV RANDOM WALKS 1 

By Cheng-Der Fuh 

Academia Sinica 

Let {X n ,n > 0} be a Markov chain on a general state space X 
with transition probability P and stationary probability ir. Suppose 
an additive component S n takes values in the real line R and is ad- 
joined to the chain such that {(X n , S n ),n > 0} is a Markov random 
walk. In this paper, we prove a uniform Markov renewal theorem 
with an estimate on the rate of convergence. This result is applied to 
boundary crossing problems for {(X„,S„),n > 0}. To be more pre- 
cise, for given b > 0, define the stopping time r = r(6) = inf {n : S n > 
b}. When a drift n of the random walk S n is 0, we derive a one-term 
Edgeworth type asymptotic expansion for the first passage probabil- 
ities Ptv{t < m} and P^{r < m, S m < c}, where m < oo, c < 6 and P^ 
denotes the probability under the initial distribution n. When n^0, 
Brownian approximations for the first passage probabilities with cor- 
rection terms are derived. Applications to sequential estimation and 
truncated tests in random coefficient models and first passage times 
in products of random matrices are also given. 



1. Introduction. Let {X n ,n > 0} be a Markov chain on a general state 
space X with c-algebra A. Suppose an additive component S n = J2t=o^,t 
with So = £o = 0, taking values in the real line R, is adjoined to the chain 
such that {(X n , S n ),n > 0} is a Markov chain on^xR with 

P{(X n ,S n )eAx {B + s)\(X n _ 1 ,S n - 1 ) = (x,s)} 
(1.1) =P{(X u Sx) eAx B\(X ,S ) = (x,0)} 

= P(x,Ax B), 
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for all x € X,s €Tl,A £ A. and B £ B (:= Borel <r-algebra on R). The chain 
{(X n ,S n ),n > 0} is called a Markov random walk. For an initial distribu- 
tion v on Xq, let P v denote the probability measure under the initial dis- 
tribution v on Xq and let E v denote the corresponding expectation. If v is 
degenerate at x, we shall simply write P x (E x ) instead of P u (E u ). In this 
paper, we shall assume that {X n ,n > 0} has an invariant probability tt. 
For b > 0, define the stopping time 

(1.2) r = r(6)=inf{n:5 n >6}, r+ = r(0). 

In a variety of contexts, for given m < oo and c < b, we need to approximate 
the first passage probabilities 

(1.3) P n {r<m}, 
and 

(1.4) P n {T<m,S m <c}. 

It is known that, with some proper identifications, (1.3) is the probability 
that the waiting time for the (m — l)th customer in a single server queue 
exceeds b; it is also the probability of ruin in finite time in risk theory [cf. 
Asmussen (1989a, b, 2000)]. The joint probability of r and S m in (1.4) is an 
important ingredient to study truncated test in random coefficient models. 
Note that discrete time ARCH model can be defined, with some modifica- 
tions, in the framework of random coefficient models; compare Bougerol and 
Picard (1992). 

When the increments £j of the random walks are independent and identi- 
cally distributed (i.i.d.) random variables. Siegmund (1979, 1985) and Sieg- 
mund and Yuh (1982) developed a so-called "corrected Brownian approxi- 
mation" by computing correction terms in the Brownian approximation to 
approximate the first passage probabilities (1.3) and (1.4). In the case of 
a finite state ergodic Markov chain, Asmussen (1989b) derived a first-order 
corrected Brownian approximation for one-barrier ruin problems in risk the- 
ory, while Fuh (1997) studied one-barrier and two-barrier boundary crossing 
probabilities, and derived a second-order corrected Brownian approximation 
in Markov random walks. Arndt (1980) studied asymptotic properties of the 
distribution of the supremum of a random walk on a Markov chain. Mali- 
novskii (1986) derived asymptotic expansions in the central limit theorem of 
(1.4) for Harris recurrent Markov chains. For a general account on ruin prob- 
abilities, the reader is referred to Asmussen (2000) and references therein. 

In this paper, we study asymptotic approximations of the first passage 
probabilities (1.3) and (1.4) for Markov random walks on a general state 
space. The limiting behavior for (1.3) and (1.4) is defined as m — > oo, and 
b = Qm 1 / 2 and c = 7m 1 / 2 for some 7 < Q > 0. When a drift /i of the random 
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walk S n is zero, we derive one-term Edgeworth type asymptotic expansions 
of (1.3) and (1.4). In the case of fi ^ 0, we first define the conjugate trans- 
formation of the transition probability in Markov random walks and then 
derive corrected Brownian approximations for the first passage probabili- 
ties (1.3) and (1.4). Motivated by the approximations of (1.3) and (1.4), 
we study a uniform Markov renewal theorem including a rate of conver- 
gence. There are three aspects to provide the uniform Markov renewal the- 
orem. To begin with, the condition of uniform ergodicity with respect to 
a given norm considered in this paper is different from the previous one [cf. 
Kesten (1974), Athreya, McDonald and Ney (1978), Shurenkov (1984, 1989), 
Alsmeyer (1994) and Fuh and Lai (2001)] and will be applied to products 
of random matrices. Second, we study the Markov renewal theorem with an 
estimate on the exponential rate of convergence. Early work in the respect 
can be found in Silvestrov (1994), using coupling. When the increments £t 
of the random walks are i.i.d. random variables, rates of convergence for the 
renewal theorem can be found in Stone (1965a, b), Carlsson and Wainger 
(1982), Carlsson (1983), Kartashov (1996) and Kovalenko, Kuznetsov and 
Shurenkov (1996). Third, the renewal theorem is in a uniform version in the 
sense of varying drifts. Uniform renewal theorems for simple random walks 
have been studied extensively in the literature; the reader is referred to Lai 
(1976), Kartashov (1980), Zhang (1989) and Silvestrov (1978, 1979, 1995), 
and references therein. 

The remainder of the paper is organized as follows. In Section 2, we for- 
mulate the problem and state our main results: a uniform Markov renewal 
theorem with rate of convergence; one-term Edgeworth type asymptotic ex- 
pansions for the first passage probabilities (1.3) and (1.4) in the case of zero 
drift; and corrected Brownian approximations for the first passage proba- 
bilities (1.3) and (1.4) when fi ^ 0. The proofs are given in Sections 4-6, 
respectively. Applications to sequential estimation and truncated tests in 
random coefficient models and first passage times in products of random 
matrices are in Section 3. 

2. Main results. Let {(X n ,S n ), n > 0} be a Markov random walk on 
X x R. For ease of notation, write P(x,A) = P{x,A x R) as the transi- 
tion probability kernel of {X n ,n > 0}. For two transition probability kernels 
Q(x, A),K(x, A), x £ X, A £ A and for all measurable functions h(x),x £ X, 
define Qh and QK by Qh(x) = J Q(x, dy)h{y) and QK(x, A) = J K(x, dy)Q(y, A), 
respectively. 

Let M be the Banach space of measurable functions h:X — ► C (:= the 
set of complex numbers) with norm \\h\\ < oo. We also introduce the Banach 
space B of transition probability kernels Q such that the operator norm 
IIQH = sup{||Qg||; ||^|| < 1} is finite. Two prototypical norms considered in 
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the literature are the supremum norm and the L p norm. Two other com- 
monly used norms in applications are the weighted variation norm and the 
bounded Lipschitz norm, described as follows: 

1. Let w:X^> [l,oo) be a measurable function, define for all measurable 
functions h, a weighted variation norm \\h\\ w = sup x£X \h(x)\/w(x), and 
set M w = {/i:||/i|| u ,<oo}. The corresponding norm in B w is of the form 
\\Q\\ W = sup x . gA . / |Q| (x, dy)w(y)/w(x). 

2. Let (X,d) be a metric space. For any continuous function h on X, the 
Lipschitz seminorm is defined by ||/i||l '■= su P x ^y\ a ( x ) ~ a (y)\/d(x, y). 
The supremum norm is ||/i||oo = sup xg ^ \h(x)\ . Let ||/i||bl := IHIl + Halloo 
and A/bl = {h : ||^||bl < °°}- Here BL stands for "bounded Lipschitz" and 
Abl is the Banach space of all bounded continuous Lipschitz functions 
on X. 

Denote by P n (x,A) = P{X n E A\Xq = x}, the transition probabilities 
over n steps. The kernel P n is an ra-fold power of P. Define also the Cesaro 
averages P^ = Y^=qP^ / n i where P° = P^ = I and I is the identity oper- 
ator on B. 

Definition 1. A Markov chain {X n ,n > 0} is said to be uniformly 
ergodic with respect to a given norm || • ||, if there exists a stochastic ker- 
nel II such that -> n as n — > 00 in the induced operator norm in B. 
The Markov chain {X n ,n > 0} is called -w-uniformly ergodic in the case of 
weighted variation norm. 

The Markov chain {X n ,n > 0} is assumed to be irreducible [with respect 
to a maximal irreducible measure ip on (Af,«4)], aperiodic and uniformly 
ergodic with respect to a given norm || • ||. In this paper, we assume ip is a- 
finite, and {X n ,n > 0} has an invariant probability it. It is known [cf. Theo- 
rem 13.3.5 of Meyn and Tweedie (1993)] that for an aperiodic and irreducible 
Markov chain, if there exists some A-small set C and some P°°(C) > such 
that, as n — > 00, 



/ X c (dx)(P n (x,C)-P°°(C))^0, 
Jc 



where Ac(-) = A(-)/A(C) is normalized to a probability on C, then the chain 
is positive, and there exists a (/9-null set N such that, for any initial distri- 
bution v with v(N) = 0, 



u(dx)P n (x,-) -7T 



as n — > 00, 



where || • ||t v denotes the total variation norm. Theorem 1.1 of Kartashov 
(1996) gives that P has a unique stationary projector U in the sense that 
n 2 = n = Pn = LLP, and II(x, A) = tt(A) for all x £ X, A G A. 
The following assumptions will be used throughout this paper. 
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CI. There exists a measure $ on X x R, and measurable function h on X 
such that j-K{dx)h(x) > 0, ^(X x R) = 1, J^>(dx x R)/t(x) > 0, and 
the kernel T(x, A x B) = P(x,A x B) — h{x)^(A x B) is nonnegative 
for all A G .A and, B £ B. 

C2. For all ifA', sup||&||<i ||-E[/j(Xi)|X = x]|| < oo. 

C3. sup^^l^il 2 < oo and, for all xeX, supi^n^ ||£ , [|^i| r /i(Xi)|A" = x]\\ < 
oo for some r > 1 . 

C4. Let be an initial distribution of the Markov chain {X n ,n > 0}; assume 
that for some r > 1, 



(2.1) 



sup 

||h||<l 



< oo. 



C5. Assume that for some no > 1, /f^ J^g^ |£'a:{exp(i6'^i)}| no 7r(dx) (i# < oo. 
C6. There exists C R containing an interval of zero such that, for all 

x G X and 6 £ 9, sup^n^ ||^[exp(^i)/i(Xi)|X = x}\\ < C < oo, for 

some C > 0. 

C7. There exists a u-finite measure M on (Af,^4) such that, for all x G 
Af, the probability measure P x on (Af,^4) defined by P x (-A) = P(X\ G 
^4|Xo = x) is absolutely continuous with respect to M, so that P X (A) = 
J A p(x,y)M(dy) for all A G A, where p(x, •) = dP x /dM. 



Remark 1. Condition CI is a mixing condition on the Markov chain 
{(X n , S n ), n > 0}. It is also called a minorization condition in Ney and Num- 
melin (1987), where they constructed a regeneration scheme and proved 
large deviation theorem. An alternative condition for CI is that there ex- 
ists a measure ^ on X, and family of measures {h(x,B);B G B} on R, for 
each x G X such that the kernel T(x, AxB) = P(x, Ax B) - h(x, B)^f(A) is 
nonnegative for all A G A and B G B. If a Markov chain is Harris recurrent, 
then CI holds for n-step transition probability. It is known that under the 
irreducible assumption, CI implies that {(X n ,£ n ),n > 0} is Harris recurrent 
[cf. Theorem 3.7 and Proposition 3.12 of Nummelin (1984) and Theorem 
4.1(iv) of Ney and Nummelin (1987)]. An example on page 9 of Kartashov 
(1996) also shows that there exists a uniformly ergodic Markov chain with 
respect to a given norm, which is not Harris recurrent. Theorem 2.2 of Kar- 
tashov (1996) states that under condition CI, a Markov chain {X n ,n > 0} 
with transition kernel P is uniformly ergodic with respect to a given norm 
if and only if there exists < p < 1 such that 

(2.2) ||p n -n|| =0(p n ), 

as n — > oo. When the Markov chain is uniformly ergodic with respect to the 
weighted variation norm, (2.2) still hold without condition CI. 
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Remark 2. Conditions C2-C4 are standard moment conditions. Con- 
dition C6 implies that the exponential moment, in the sense of the corre- 
sponding norm, of £1 exists for 6 in 0. Condition C5 implies that for all 
n > no, S n has a bounded probability density function for given X n . The 
existence of the transition density in C7 will be used in Theorems 2 and 3 
only. It holds in most applications. 

Next, we will describe the uniform version of conditions C1-C7. Since 
the uniform version is in the sense of varying drift [see (2.13)], we consider 
a compact set T C R which contains an interval of 0. For each a £ T, let 
{(X® , Sn),n > 0} be the Markov random walk on a general state space X 
defined as (1.1), with transition probability P a and invariant probability 
measure 7r a . For each a S T, the Markov chain {l",n > 0} is assumed to 
be irreducible [with respect to a maximal irreducible measure ip on (^f,^4)], 
aperiodic and uniformly ergodic with respect to a given norm || • ||. 

To establish the uniform Markov renewal theorem, we shall make use of 
the uniform version of (1.1) in conjunction with the following extension of 
the uniform Cramer's (strong nonlattice) condition: 

(2.3) g{6) := inf inf 11 - E°{exp(ivS?)}\ > for all 9 > 0. 

aer \v\>$ 

Additive component S n is called strongly nonlattice if F has only one ele- 
ment. In addition, we also assume that the [conditional uniform] Cramer's 
(strong nonlattice) condition holds. There exists m > 1 such that 

(2.4) suplimsup|£ a {exp(i0S^)|X o ,X m }| < 1. 

oer |0|->oo 

Note that under condition C5, (2.3) and (2.4) can be removed. Next, we 
assume the strong mixing condition holds. There exist 71 > and < p\ < 
1 such that, for all k > and n > 1, and for all real- valued measurable 
functions g, h with g,h£ M, 

\E u {g(X k )h{X k+n )} - {E v g{X k )}{E v h{X k+n )}\ <7iPi _1 - 

Note that when the norm is the weighted variation norm, we only need that 
(2.3) and (2.4) hold without the strong mixing condition. 
For a 6 T, the uniform versions of C1-C6 are: 

Kl. There exists a measure on X x R, and measurable function h on X 
such that / TT a (dx)h(x) > 0, ^ a {X x R) = 1, / ^ a (dx x R)h(x) > 0, and 
the kernel T a (x, AxB) = P a (x, AxB) — h(x)^ a (A x B) is nonnegative 
for all A e A and B £ B. 

K2. For allxeA', sup aer sup^n^ \\E a [h(Xf )\X§ = x]\\ < 00. 

K3. sup Qgr sup^, £^|£f| 2 < 00 and for all x G X, sup Qgr sup||^|| <1 ||-E a [|£f | r x 
h{Xf )\X% =x}\\ < 00 for some r > 1. 
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K4. Let v a be an initial distribution of the Markov chain {X^,n > 0}; as- 
sume that for some r > 1, 



sup sup 

a&T \\h\\<l 



x&X 



h(x)E*\£*\ r v a {dx) 



< oo. 



K5. Assume that for some no > 1, 



sup 

aer J—oo Jx&x 



\E^{exp(ie^)}\ n °7r a (dx) d6 < oo. 



K6. There exists C R containing an interval of zero such that, for all 



x € X and 6 6 G, sup Qer sup^n^ ||E a [exp(0ff )h(Xf )\X L Q 
for some C > 0. 



x} \ < C 



Theorem 1. Let {(X",S"),n > 0} be a uniformly strong nonlattice 
Markov random walk satisfying K1-K4 with r > 2 in K3. Let fi a := fif : = 
££ff > and /xg := ££(ff ) 2 < oo. T/ien, as s ^ oo, 

oo 

(2.5) £i?{s<,S£ + €4} 

n=0 

oo 

(2.6) £i?{-oo<S£ A} 

n=0 

Furthermore, i/K6 holds, then for some r\ > 0, as s — > oo, 

(2.7) E < 3? < ^ 6ii}=(jJ + ^jl)^^) + °( e " riS )- 

Remark 3. When T has only one element and the increments £ n are 
i.i.d., these results are proved via Fourier transform and Schwartz's theory 
of distributions. These methods can be extended to the Markov case via 
perturbation theory of the transition probability operator. Such extensions 
can also be modified to yield the rate of convergence in Markov renewal 
theory, which generalizes the corresponding results of Stone (1965a, b) and 
Carlsson (1983) for simple random walks. 

Remark 4. For each fixed a, the Markov renewal theorems in Theo- 
rems 1-4 in Fuh and Lai (2001) provide the rate of convergence as s — > oo. 
However, in the applications to Theorem 3, we shall be letting a — > simul- 
taneously with s — > oo. Consequently, we must consider the possibility that 
certain unpleasant situations might occur, such as a case in which, as a — ► 0, 
the rate of convergence to of the error terms in (2.5)-(2.7) gets slower and 
slower. Theorem 1 guarantees that this cannot happen; that is, that there 
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is a certain rate of convergence which applies uniformly to all a in some 
neighborhood of 0. 

Let v be an initial distribution of the Markov chain {X n ,n > 0} and let 
fi = E n £x, a 1 = lim^oo n~ l E v {(S n - nfi) 2 } and k = lim n ^oo n~ 1 E u {(S n - 
n[i) 3 }, which are well defined under C4 for some r > 3. It will be convenient 
to use the notation 

(2.8) P^ s) {A) = P u {A\S m = s}. 

Let r_|_ = inf{n > 1 : S n > 0} be the first ascending ladder epoch of S n , 
r n = inf{/c > T n -\ : Sk > <SV„_i} be the nth ascending ladder epoch of S n , for 
n = 2, 3, . . . , and let r_ = inf {n > 1 : S n < 0} be the first descending lad- 
der epoch of S n . Since \i > 0, T n are finite almost surely under the probabil- 
ity P{X T+ £ A\Xq = x} and therefore, the associated ladder heights S Tn are 
well-defined positive random variables. Furthermore, {(X Tn , S Tn ),n > 0} is 
a Markov chain, and it is the so-called ladder Markov random walk. When 
fj, = 0, we can still define the ladder Markov chain via the property of uniform 
integrability in Theorem 5 of Fuh and Lai (1998). It is assumed throughout 
this paper that P x {t+ < oo) = 1 for all x G X and that the ladder random 
walk is uniform ergodic with respect to a given norm. The moment conditions 
C2-C4 and C6 for the ladder random walk are in Lemma 1 and Lemma 14, 
respectively. The uniformly strong nonlattice for the ladder random walk is 
in Lemma 13. Since C5 holds in Theorems 2 and 3, we do not need (2.4) 
anymore. Let tt + denote the invariant measure of the kernel P+(x, A x R rf ) 
which is assumed to be irreducible and aperiodic. The property of Harris re- 
current for ladder Markov chains has been established in Alsmeyer (2000). 
Therefore, CI holds for the ladder random walk. In Section 3, we will show 
how uniform ergodicity of the ladder chain and finiteness of moments of S T+ 
can be established in some interesting examples. 



Theorem 2. Let {(X n , S n ),n > 0} be a Markov random walk satisfying 
C1-C5 and C7 with r = 3 in C3. Suppose /j, = 0, cr = 1, and that there exists 
e > such that mf x P^iCi < s\Xi = x] > 0. Let b = (m 1 / 2 and s = (om 1 / 2 
for some ( > and — oo < Co < C- Then, asm-> oo, 

Pi m ^{r<m} 

(2.9) * X S 

= exp{-2(6 + p+)(6 + /3 + - s - «/3)/(m + ks/3)} + o(m 1/2 ), 

where p+ = E 7T+ S 2 + /2E n+ S T+ . If in addition c = 7m 1 / 2 for some 7 < (, then, 
usm-> oo ; 

(2.10) PAr < m, S m < c} = $ + o{m ^ 2 ), 
where $ denotes the standard normal distribution function. 
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Remark 5. Approximations (2.9) and (2.10) are the corresponding re- 
sults for Brownian motion with drift 0, 6 replaced by b + p+, s(c) re- 
placed by s + k/3 (c + k/3) and m replaced by m + ks/3. Also, note that 
the constant p + in (2.9) and (2.10) reduces to ES^ + /2ES T+ when S n is 
a simple random walk [cf. Siegmund (1985), pages 220 and 221]. Since 
P n {r < m} = P- K {S m > b} + P n {T < m,S m < b}, one-term Edgeworth ex- 
pansion of P w {S m > b} and (2.10) give a representation of (1.3). 

To state Theorem 3, we need to define a twist transformation of the 
transition probability operator, and this requirement leads us to study the 
perturbation theory of certain linear operators on M. For z G C, define linear 
operators P^, P, and Q on M by 

(2 11) ( P ^)( x ) = ^[M*iK a |*o = z], (Ph)(x)=E[h(X 1 )\X = x], 
v*h = E u {h(X )}, Qh = J h(y)TT(dy). 

When the norm is supremum norm and £ n = g(X n ), Nagaev (1957) and 
Jensen (1987) have shown that there exists sufficiently small 5 > such 
that, for \z\ < 5, N = N\{z) ®J\f 2 (z) and 

(2.12) P z Q z h = \{z)Q z h for all he M, 

where ftfi(z) is a one-dimensional subspace of M, \{z) is the eigenvalue of P z 
with corresponding eigenspace N\(z) and Q z is the parallel projection of J\f 
onto the subspace N\{z) in the direction of A/2(z). Extension of their argu- 
ment to weighted variation norm and random £ n satisfying some regularity 
assumptions is given in Fuh and Lai (2001). We extend this result to uniform 
ergodic Markov random walks with respect to a given norm in the Appendix. 
Let h\ G J\f be the constant function h\ = \ and let r(x;z) = (Q z hi)(x). 
From (2.12), it follows that r(-;z) is an eigenfunction of P z associated with 
the eigenvalue X(z), that is, r(-;z) generates the one-dimensional eigenspace 

M(*). 

In particular, z = a£R such that there exists 5 > and a G [—5, 5] :=T. 
Define the "twisting" transformation 

(2.13) P a (x, dy x ds) = r ^ ma \ e~ k ^ +as P{x, dy x ds) where A = log A. 

r(x; a) 

Then P a is the transition probability of a Markov random walk {(X%,S%), 
n > 0}, with invariant probability 7r a . The function A(a) is normalized so 
that A(0) = AW(0) = 0, where C 1 ) denotes the first derivative. Then P° = P 
is the transition probability of the Markov random walk {(X n ,S n ),n > 0} 
with invariant probability it. Here and in the sequel, we denote as the 
probability measure of the Markov random walk {(X",S"),n > 0} with 
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transition probability kernel (2.13), and having initial distribution v a . For 
ease of notation, we denote v a := u, and let E° be the expectation under 



It is known that A is a strictly convex and real analytic function for 
which AW(a) = E%£f. Therefore, E%£f <,=,or > 44> a <,=,or > 0. For 
any value and \a\ < 5, there is at most one value a' with \a'\ < 5, 

necessarily of opposite sign, for which A(a) = A(a'). Assume such a' exists; 
we may let that Qo = min(a, a') and a± = max(ct, a') such that ao < < a\ 
and A(«o) = A(ai). Denote A = a± — Qo- We also assume, without loss of 
generality, that a 2 = A^ 2 ^(0) = 1, where ^ denotes the second derivative. 

Theorem 3. Let {(X n , S n ),n > 0} be a strong nonlattice Markov ran- 
dom walk satisfying C1-C5 and C7. Let {(X^,S^),n> 0} be the Markov 
random walk induced by (2.13). Suppose there exists £ > such that ini x ^{£1 <• 
— e\Xi = x} > 0. Let b = £m 1//2 for some ( > 0, c = 7m 1 / 2 for some 7 < £, 
and also that y/mA = 5 is a fixed positive constant. Then as m — ► 00, for 
j = or 1, 



Remark 6. Let M„ = r(X n ; a) ex.p{aS n — nA(a)} and T n be the o- 
algebra generated by {(X t ,St),t < n}. Then for \a\ < 5, {M^^F^n > 0} is 
a martingale under any initial distribution u of Xq [cf. Ney and Nummelin 
(1987) and Fuh and Lai (1998)]. Note that r(-; a) in (2.14) reduces to 1 when 
S n is a simple random walk. Hence, r(y;a)/r(x;a) can be regarded as the 
reflection of Markovian dependence under uniform ergodicity condition with 
respect to a given norm. 

3. Examples. In this section, we give examples of strongly nonlattice 
Markov random walks satisfying conditions C1-C7, and such that the un- 
derlying Markov chain {X n ,n > 0} is irreducible and aperiodic. Many time 
series and queuing models X n are irreducible, aperiodic and w-uniformly er- 
godic Markov chains, as shown in Chapters 15 and 16 of Meyn and Tweedie 



P"A{T<m,S% <c} 

r r(X T+ ;aj) r(x;ai-j) 
Jxex r(x;ctj) r(X T+ ;oti-j) 



(2.14) 



x exp[-{-iyA(b + P+ )]ir a i{dx) 



+ o(m 
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(1993), and conditions C2-C4 and C6 are moment conditions on the addi- 
tive components attached to X n that are satisfied in typical applications. 
However, renewal theorems are often applied to the ladder random walk, as 
in Section 2. The techniques used by Meyn and Tweedie (1993) to prove 
the tii-uniform ergodicity of a rich class of time series and queuing models 
can also be applied to show that their ladder random walks indeed satisfy 
conditions C1-C7, as illustrated by the following examples. 

3.1. Random coefficient models. Let {X n ,n > 1} be the Markov chain 
which satisfies a first-order random coefficient autoregression model 

(3.1) X n = f3 n X n _i + e n , Xq = 0, 

where ((3 n )n>i 1S a sequence of i.i.d. random variables with E(3 n = f3 and 
Var(/3 n ) = a 2 , where a > is known. (£ n )n>i is a sequence of i.i.d. random 
variables with Ee n = and Var(e n ) = 1. Further, we assume that ((3 n )n>i 
and (e n ) n >i are independent, and ((3 n ,£ n )' has common density function q 
with respect to Lebesgue measure is positive everywhere. 

In the case of AR(1) model for which f3 n is a constant (3, Lai and Siegmund 
(1983) proposed a sequential estimation procedure for the unknown param- 
eter j3. Pergamenshchikov and Shiryaev (1993) generalized their results to 
the model (3.1). They introduced the stopping time 

(3.2) r = T c = infL>l:f: T -5^>c}, 



k=l 




where c> is a fixed number, and considered a modification of the sequential 
least-squares estimate 

(3.3) b T -- 

Their Theorems 1-3 showed that T c < oo with probability 1 for any c > 0, 
and is asymptotically normal under some moment conditions. 

In this section we investigate the limiting behavior of T c under the sta- 
bility assumption (3 2 + a 2 < 1. Meyn and Tweedie [(1993), Theorem 16.5.1] 
established w(x) = |x| 2 -uniform ergodicity of the random coefficient model 
(3.1) by proving that a drift condition is satisfied. By Lemma 15.2.9 of Meyn 
and Tweedie (1993), it is also (\x\ + l)-uniform ergodic. Suppose the condi- 
tional distribution of £ n = X 2 /(l + a 2 X 2 ) given Xq, ■ ■ ■ ,X n is of the form 
Fxn-uXn such that 



(3.4) limsup 

|0|->oo 



e m dF Xil3x+z (0 \q(P,z)df3dzir(dx] 

X J~ oo J — oo LJ— oo 



<1, 
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where ir is the stationary distribution of {X n }. Since £1 has probability 
density function with respect to Lebesgue measure, (2.4) can be removed. Let 
S n = X^r=o £« • Then { (X n , S n ) , n > 0} is strongly nonlattice, and by Theorem 
6(h) of Fuh and Lai (1998), so is the ladder random walk with transition 
kernel P + defined by 

(3.5) P+{x, AxB) = P{X T+ G A, S T+ G B\X = x}. 
Assume furthermore that 

(3.6) supEa.{^i(l + |/3i| + |ei|)}<oo and \i := > 0. 

X 

We first note that X T+ has a positive density function with respect to 
Lebesgue measure C and is therefore ^-irreducible. Moreover, the ladder 
chain is clearly aperiodic; see Section 5.4.3 of Meyn and Tweedie (1993). 
Let w{x) = \ x\ + 1. To show that the ladder chain is ^-uniformly ergodic, by 
Theorem 16.0.1 of Meyn and Tweedie (1993), it suffices to show that there 
exist positive constants b, A and a petite set C such that 

(3.7) E x w{X T )-w(x)<-\w(x)+bt c {x) for all x G R, 

for T = r+. We first show that the drift condition (3.7) in fact holds for all 
stopping times T [with respect to the filtration generated by (X n ,S n )] such 
that, for some a > 0, 

(3.8) E X T < a(\x\ + 1) for all x G R. 

We then show that r + satisfies (3.8) and therefore (3.7) indeed holds for 
T = r + . 

Let B > 0, and denote e* = £il{| £i |<m> £** = ^i^-{\ £i \>B}- Note that 
E\X T \=E\0T---p 1 Xo + &r- 1 ---Piei + ---+p 1 eT-i + eT\ 

(3-9) 

< \p\ \x\ + (1 - \(3\)- l B + for X = x. 

i=l 

Since the £** are i.i.d. random variables, Wald's equation yields 

T 



(3.10) E x 



E 

1 = 1 



c ; 



(^T)S|ef | < a (\x\ + l)E\el* 



by (3.8). Since S|ei| < oo from (3.6), we can choose B sufficiently large 
so that a-E|eJ* | + \(3\ < 1. In view of (3.9) and (3.10), we can then choose 
0<A<1- \p\ -aE\e\*\, b > (1 - \(3\)^B + aE\ef \ and C = {x:\x\ < K} 
with K sufficiently large such that (3.7) holds. Note that C is a petite set; 
see Section 5.2 of Meyn and Tweedie (1993). 
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To show that r+ satisfies (3.8), we use Wald's equation for Markov random 
walks [see Lemma l(i) in Section 4]: For any stopping time T with E V T < oo 
and E u w{Xt) < oo, 

(3.11) E U S T = iiE v T + E V {A{X T ) - A(X )}, 

where sup x \A(x)\/w(x) < oo, and A is defined in (4.1). Let Q = £ilLr&<B}> 

S ( n B) = C[ B) + --- + & B) and t(B) = inf{n : S { n B) > 0}. Since fi (= E^ = 
E n Xi/(l +a 2 Xi)) > 0, we can choose B large enough such that (= 

-^ttCi ) > 0. Note that Sk < S n and t(B) > r+. Hence it suffices to show 
that r(-B) satisfies (3.8). By the monotone convergence theorem, we need 
only show that (3.8) holds with T = r(B)Am for every m > 1. Since S T m)Am < 
B, (3.11) yields 

B>/i^r-^|A(X T )|-|A(x)| 

(3.12) 

> M (B) ^r - cE x \X T \ - c{\x\ + 2), 

since |A(x)| < cw(x) = c(\x\ + 1) for some c > and all x. By (3.9) and 
(3.10), 

E x \X T \<\f3\\x\ + (l-\(3\)- l B 

(3.13) 

+ {E x T)E{\e l \t { \ ei>B} ). 

Choosing B large enough so that C-E , |ei|l{| £l | > ^} < [1^/2, we obtain from 

(3.12) and (3.13) that 

B + c(l - l/?!)^ 1 ^ + 2c + c|x|(l + |/?|) > fi {B) E x T/2, 

proving (3.8) for T = r(5) Am. Note that the ladder chain also satisfies the 
mixing condition CI by Theorem 1 of Alsmeyer (2000). 

From (3.9) and (3.10) with T = r+, it follows that sup x {E x w(X T+ ) / 
w(x)} < oo. Hence C2 holds for the ladder chain. To prove the moment 
condition C6 hold for the kernel (3.5), we assume the additional moment 
conditions 

(3.14) sup£' x exp{6'(^i + /?i + ei)}<oo for 6 G 9. 

X 

First note that E x exp{6S T+ }w{X T+ ) < {E x exp{6pS T+ }} l / p {E x w q (X T+ )}V«, 
where p~ 1 + q~ 1 = 1. Lemma 14 in Section 6 implies that E x exp{6pS T+ } < oo 
for x £ X. From (3.9), there exists a constant c q depending only on q such 
that 

E x \X T+ \^<c q ^\x\ q + (l-\(3\- 1 )E x max\e i \^ 
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<c q \\x\9 + (l-\0\- 1 )E x ^2\e i \ 



i=l 



Cg{N«+(l 



)E\e l \ q E x r + }. 



It follows that sup x E x (exp{6S T+ }w(X T+ ))/w(x) < oo. By using the same 
argument and applying Wald's equation for Markov chain in Lemma 1, it 
follows that the moment condition C3 also holds for the ladder chain. Since 
So = and sup x E x exp{# S T+ } /w(x) < oo, C4 also holds for the kernel P + 
if the initial distribution u satisfies $™ \x\dv[x) < oo. Hence C1-C6 are 
satisfied by the ladder random walk with transition kernel P+ when the 
underlying chain is the random coefficient model (3.1) and {£ n } satisfies 
(3.4), (3.6) and (3.14). 

Under the normality assumption on ((3k, Sk) with known a 2 , the log- 
likelihood ratio statistic Z n for testing Hq : (3 < [1q against H\ : /3 > fio is 
given by 



(3.15) 



^ n ~ h y~]((Xj — niXi^i) 2 — (Xi — fioXi-i) 



1=1 



where [i\ is so chosen fi\ > hq. Define the stopping time T\ = inf{n > 1 : Z n > A}. 
Given m > 0, we consider the test of Hq : (3 < /zo against H\ : /3 > fio defined 
by the following: stop sampling at min(Tx,m); reject Hq if T\ < m, and 
otherwise do not reject Hq. 

Under the stability assumption [3 2 + a 2 < 1 , and consider Y n = (X n -\, X n ) 
as the underlying Markov chain. Suppose the conditional distribution of 
in = [X n - /?iX n _i) 2 - (X n - (3QX n _i) 2 given Y ,...,Y n is of the form F Yn _ lt Y n 
such that 



(3.16) 



lim sup 

l*l-o 



XxX J-oo 



oo poo 



e^dF y ^ y+z (0 



x q(P, z) dp dz 7Ti (dy) 



<1, 



where tt\ is the stationary distribution of {^n}. Let S n = 5Zi=o&i then 
{(Y n , S n ),n > 0} is strongly nonlattice. It is easy to see that there exists 
e > such that inf^ -P^iICi < —£\X\ = x] > 0. The rest of the argument is 
the same as (3.4)-(3.14) and is omitted. 



3.2. Products of random matrices. In this section, we apply the renewal 
theory in Section 2 to generalize some results of Kesten (1973) on prod- 
ucts of random matrices in three directions. First, while Kesten considered 
products of i.i.d. matrices M n , we work with the more general setting in 
which {(X n ,M n ),n > 0} are products of Markov random matrices. Second, 
we provide uniform renewal theory, with polynomial and exponential rate of 
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convergence, respectively. This extension enables us to apply corrected dif- 
fusion approximation for the first passage probabilities. Third, while Kesten 
assumed the entries of M n to be positive with probability 1, we can dispense 
with this assumption. Moreover, our proof is considerably simpler and pro- 
vides a more transparent description of the basic constants that appear in 
his results. 

We shall consider k X k nonsingular matrices M, with real entries, and de- 
fine the norm by ||M|| = sup| x | =1 |Mx|, where | • | is a norm in R fc . Following 
Bougerol ( 1988 ) , define x( M ) = max(log ||M||,log ||M _1 ||). Let {(X n ,M n ),n > 
0} be a Markov chain satisfying the following assumptions: 

(Al) M n is a k x k nonsingular matrix with real entries such that 

sup£'{exp(ax(Mi))|Xo = x} < oo for some a > 0. 

X 

(A2) {X n ,n > 0} is a ^-uniformly ergodic Markov chain and satisfies C7. 
(A3) {(X n , M n ), n > 0} is quasi-irreducible and 71 7^ 72, where 71 > 72 > • • • > 7fc 
denotes its Lyapunov exponents. 

For the definition of "quasi-irreducibility," see Bougerol [(1988), page 199]. 
For the definition and basic properties of Lyapunov exponents, see Bougerol 
and Lacroix [(1985), Sections III. 5 and III. 6) and Bougerol [(1988), pages 
197 and 198]. Let Mq be the k x k identity matrix and define the product 

(3.17) U n = M n ---M . 

Let u be unit column vectors in R fc . Since LT n is nonsingular, H n u and 

n 

(3.18) log|II n u|=X> where £ t = log(|IlH/|n^H). 

t=i 

Let Yq = (Xq, u), . . . , Y n = (X n ,H n u/\H n u\). Define £t as (3.18), and let 
S n = J2t=i Ct- Then it follows from (3.17) that {(Y n ,S n ),n > 0} is a Markov 
random walk. Under (Al)-(A3), Bougerol (1988) has shown that Y n has 
an invariant measure and is uniform ergodic with Holder continuous norm 
[see Definitions 3 and 4 of Bougerol (1988)]. Moreover, in view of (Al), 
conditions C3 and C4 are satisfied for every r > 1. Furthermore, condition 
C6 holds. Assuming £1 to be strongly nonlattice and conditional strongly 
nonlattice, we can therefore apply the renewal theorems in Section 2 to the 
Markov random walk {(Y n ,S n ),n > 0}, thereby both generalizing Kesten's 
(1973) renewal theory for products of i.i.d. matrices with positive entries 
and providing convergence rates in the renewal theorems. The mean value 
\x in these theorems is equal to 71 of upper Lyapunov exponents. 

Let S denote the sphere consisting of unit column vectors in R fc . For 
u G S, define the stopping time 

(3.19) N(b) = mf{n > 1 : \U n u\ > e b } = mf{n >l:S n > b}, inf = 00. 
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Suppose 71 > 0. Since sup xGX ueS E(\^i\ v \Xq = x) < oo in view of (Al), The- 
orem 2 in Section 2 can be applied to show that (2.9) and (2.10) hold for 
N(b) as b — > oo. This generalizes Theorem 2 of Kesten (1973) that considers 
the case of i.i.d. M n with positive entries. 

We next consider the case 71 < and assume in addition that, for some 
< a* < a [where a is given in (Al)], 

(A4) inf E{\M lU f \X = x} > k a * /2 . 

For i.i.d. matrices M n = (M n (h, i))i<h t<k with positive entries, Kesten's 
(1973) Theorem 3 restricts u to the subset 5+ of S consisting only of vectors 
with nonnegative entries and assumes, among other conditions, that 



(3.20) 




which implies (A4) with S replaced by 5+ [see the inequality preceding (2.66) 
in Kesten (1973)]. Under assumptions (Al) and (A2), if the assumption of 
quasi-irreducibility in (A3) is strengthened into "strong irreducibility" [see 
Section 5 of Bougerol (1988) for its definition and properties], then it can 
be shown that tilting for the operator P a is defined by 

(P a f)(x,u) = Eie^fWlYo = (x,u)} 

on the space L(a) of functions on X x S with the Holder continuous norm 
whose spectrum is taken over x £ X [if the induced Markov chain { (Y n , S n ), n > 
0} is irreducible]. By making use of (A1)-(A4) and Proposition 1 in the Ap- 
pendix, (P a f)(x,u) is well defined. Let A(a) and r((x,u);a) be the largest 
eigenvalue and associated eigenfunction defined as (2.11)-(2.13). 

Therefore, the usual tilting argument shows that Theorem 3 of Kesten 
(1973) holds. In particular, taking B = e b yields 

(3.21) BP X < max |n n -u| > B > — > Kr((x, u), a) asB^-oo, 
I n >! J 

where K = ( e - fo (A(a)-i) / 7l ) J^ xRfc l/r((sc, «); a) dn^ (x, u). 

4. Proof of Theorem 1. The ingredients we need to make the uniform 
Markov renewal theorem over the family {(X°,S°),n > 0:a G T} are pro- 
vided by Lemmas 3 and 4. The proof of Lemma 3 depends on a uniform 
upper bound for the expectation of the overshoot, which we state and prove 
in Lemma 2. To prove Lemma 2, we need Wald's equation for Markov chains, 
and moment convergence of the stopping time r(b) defined in (1.2) for 
all b > 0. These are included in Lemma 1. A version of Wald's equations 
for uniformly ergodic Markov random walks can be found in Fuh and Lai 
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(1998), where they applied the spectral theory of positive operators related 
to Markov semigroups. Fuh and Zhang (2000) first derived Poisson equa- 
tions for Markov random walks, and then applied them to establish Wald's 
equations. Here in CI and C2, we applied results in Harris recurrent Markov 
random walks to obtain (first-order) Wald's equation via Poisson equation. 

Lemma 1. Assume CI and C2 and that < := E n ^i < oo. Let u be 
an initial distribution of Xq, and let T be a stopping time such that E U T < 
oo: 



The constant E u {A(Xt) — A(Xq)} is zero when v = ir. Denote £f (£i ) as 
the positive ( negative ) part of £i . 

(ii) If sup x E x (^i) < oo, then E u r(b) < oo. 

(hi) Letp > 1. Ifsup x E x (^) < oo and sup x E x (£f) p < oo, then EyS 1 ^^ < oo. 

Proof, (i) The minorization condition CI ensures that {(X n , S n ), n > 
0} is a split chain [cf. Lemma 3.1 of Ney and Nummelin (1987)]. Under the 
irreducible assumption, it is also a Harris recurrent Markov chain. Proposi- 
tion 17.4.1 and Theorem 17.4.2 of Meyn and Tweedie (1993) give that the 
following Poisson equation 



has a solution A : X — > R for almost every x € X. Under the assumption of 
supj.-Ezd^il) < oo, E u (A(Xt) — A(Xo)) is finite. Therefore, by Corollary 1 
and Theorem 4 of Fuh and Zhang (2000), we have the proof of (i). 

To prove (ii), we choose B > such that // := E n (£i(B)) > 0, where 
= &I(6 < B). Let S' n = &(J3) + ■ ■ • + f n (5) and let N b = inf{n > 
1 : S' n > b}. Then S' n < S n and N b > r(6). For m > 0, apply (i) to N b A m; we 
have E u S' NbAm = /i'E u (N b A m) + 0(1) asra-> oo. By the monotone conver- 
gence theorem, linim^oo E v (N b Am) = E u N b . Moreover, by the definition of 
N b , SN b /\ m < b + B for all m > 1. Hence b + B > fi'E u N b — a for some a > 0, 
and therefore oo > E u N b > E u r(b). 

Finally, we prove (hi). Since < 5^(6) < ^+Cr(fe)) h follows from Minkowski's 
inequality that 



Since we have already shown that E y r{b) < oo and sup^, E x {^f) p < oo by 
assumption, it follows from (i) that 



(i) If swp x E x (\£ > i\) < oo, then 

E V S T = iiE v T + E U {A(X T ) - A(X )}. 



(4.1) 



E x A{X 1 )-A(x) = EM-E^i 
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proving the finiteness of E v S^, b y □ 

Lemma 2. Assume C1-C4 with r = 2 in C3. Suppose fj, > 0. For b>0, 
let R(b) = SV(&) — b. Then, 



(4.2) supE n R(b) < 



b>0 En€l 

When the initial distribution of Xq is v , (4.2) becomes that there exists 
a constant K > such that sup fe>0 E u R(b) < E^i^j) 2 / E^\ + K . 

Remark 7. In the case of simple random walks, the upper bound (4.2) 
was given in Lorden (1970) by pathwise integration. 

Proof of Lemma 2. For any values of £i, £2, ■ ■ ■ , the overshoot function 
{R(b);b > 0} is piecewise linear, with all pieces having slope —1. We consider 
first the case where the £'s are nonnegative. It is easy to see that, for c > 0, 

c S S 

(4.3) jf R(b) db = jf T(C) R(b) db- T(C) R(b) db = \Yj? t - \R{c 

Since for c > 0, E n T(c) is finite by Lemma 1 (ii) , the sum in (4.3) has finite 
expectation by C3 and Wald's equation for Markov random walks, and since 
the other terms are nonnegative, they also have finite expectations. Since 
R(b) > for all b, we have by Fubini's theorem and Wald's equation for 
Markov random walks in Lemma l(i) 

f E n R{b) db = \E v £\E v t(c) - \E^R(c)\ 

J 

where E n £ 2 is finite via condition C3. Note that E n {A(X T ^) - A(X )} = 
in the Wald's equation. 

By Jensen's inequality and Wald's equation for Markov random walks, 

(4.4) f E^R^db^^E^Kc + E^Ric)) - \(E n R(c)f . 

J 

It is easy to see that for all b, u > 0, E n r(b + u) < E n r(b) + E n r(u), since 
the conditional expectation of r(b+u) — r(6) given r(6) = n, Xq, -Xi,£i, • • • , X n ,£ n 
equals E w t (u — r), where ^ = £1 + • • • + £ n — b> and r(u — r) is zero if r > it, 
so that E^t(u — r) < E v t(u). It follows from Wald's equation for Markov 
random walks that E^RQ)) is a subadditive function of b and therefore 

\c{E v R(c)+ g c ) 

(4.5) <\c inf (E^Rib^ + E^Ric-b)) 

1 0<6<l/2c 

< [ l,2C {E^R{b) + E^R{c-b))db= 1° E w R(b)db. 
Jo Jo 



(c) 2 . 
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Combining (4.4) and (4.5) and rewriting, we obtain 

(4.6) {E n R{c)f + (c - E^ 2 i/n)E^R{c) - cE^l/pt < 0. 

The left-hand side of (4.6) is a quadratic in E n R(c) which is nonpositive 
only between its roots, — c and E^^/fi. Therefore, E^R^c) < E^^f/fi and 
since c is arbitrary, the proof is complete for the nonnegative case. 

The case where £i , £2 , • • • may take negative values reduces to the nonneg- 
ative case through consideration of the associated sequence of positive ladder 
variables, which forms the ladder Markov random walks {(X Tn , S Tn ),n > 0} 
defined in the paragraph before Theorem 2. We first note that, by as- 
sumption, the ladder Markov chain is uniformly ergodic with respect to 
a given norm. Next, we need to verify that conditions C1-C4 still hold for 
the associated ladder Markov chains { (X Tn , S Tn ) , n > 0} . It is known [cf . 
Theorem 1 of Alsmeyer (2000)] that if {(X n , S n ),n > 0} is Harris recur- 
rent, then the associated ladder Markov chains { ( X Tn , S Tn ) , n > 0} are also 
Harris recurrent. Under the irreducible assumption, the minorization con- 
dition CI is equivalent to Harris recurrent. Therefore the ladder Markov 
chain {(X Tn ,S Tn ),n > 0} satisfies CI. The moment conditions C2-C4 hold 
by Lemma l(iii). 

Since R(b) is pointwise the same for £i,£2> • • • and the sequence of ladder 
variables, and < S T+ < , the result for the nonnegative case implies 

swpE n R(b) < + < + 

< ^ + [(gi + ) 2 + --- + (e+) 2 ] E n {it? 

- + ■•■ + &+] E^ 

by Wald's equation for Markov random walks. 

When the initial distribution of Xq is v. Under the assumption of sup x E^il 2 < 
00, and fj, > 0. It is known that as b — > 00, E X (S T ^) — b) = E 7V+ S% + /2E n+ S T+ + 
o(l) uniformly in x G X [cf. (3.20) of Fuh and Lai (2001)]. Therefore, the 
difference between sup 6>0 E u R(b) and sup fc>0 E w R(b) is a constant K > 0, 
and the proof is complete. □ 

For z G C and a G T, define the operators P°,P,i/° and Q on N as 
(2.11). By (A.1)-(A.3) and Proposition 1 in the Appendix, we can define 
eigenvalue \ a {z) of the operator P". 

Let "7£" and "T" denote "real part of" and "imaginary part of," respec- 
tively. Denote B = [s, s + h) and let 

00 

(4.7) U^ A \B) := ]T P?{s <S«<s + h,X«£A} 

n=0 

be the renewal measure for each a G T. 
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Lemma 3. Assume the conditions of Theorem 1 hold. For k > 1, let 
/x^ := E n (£f ) k > and := sup^ E x (^f) k > 0. Then, for each positive inte- 
ger k, there exist > and fj,t, f]t < oo such that 

(4.8) ^ < inf (j% < sup ^ < /4 and n k * < inf rfc < sup n% < rf k . 

Also, there exist r% > and 5 > suc/i i/iai, for all z with TZ(z) < n and 
|X(z)| < 5, then for each positive integer k, there exists vt < oo such that, 
for all a G T, 

Finally, there exists C such that, for all a G T, and s > and h<2 in (4.7), 

U {a ' A) {B) <C. 

Proof. To prove (4.8), we only consider the first part, since the second 
part can be proved in a similar way. By the assumption of uniformly strong 
nonlattice in the form (2.3), we have for all > 



so that using the fact that \e lt — 1| < \t\ for all real t, we obtain for all a G T 
and 6 > that 



where P a (-, ■ x •) denotes the transition probability of {(X%,S%),n > 0}, and 
7r a (-) denotes the invariant probability of {(X",S"),n > 0}. This implies 
that inf aej 4/i° > supg >0 g(6)/9 > 0. Hence, we get the existence of fiu. The 
existence of /j, k * for positive integers k now follows from Jensen's inequality. 

For the upper bound in (4.8), note that since e t > t k /kl for all t > 0, we 
have by K6 that 



g{6) := MjEne*® 



l\>g(e)>0, 







k\C 



< 



for all a G T. 
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To prove the second assertion, note that by Proposition 1 and the Cauchy- 
Schwarz inequality, for TZ(z) < r\/2 and < 5, there exists c > such 

that, for all a G V, 



d k 



= \E«m) k e z ^)\ + c < E«m) k e r ^l 2 ) + c 

< [E$((& f k )E«(e^t' 2 + c < (fi* 2k C) 1/2 + c := v* k . 



The final assertion can be proved by using Lemma l(i), Lemma 2 and 
(4.8). Let A = X for simplicity; then for all a G T, s > and h < 2, there 
exists C\ < oo such that 



1 



U«'*{B)<—[E<ZS« {s+h) 



< 



1 



< 



< 



1 

1 

Mi* 



2 + sup sup E^R(s) 
aer o<s<oo 

2 + sap M!l + K 



2 + + K 

Mi* 



+ Ci:=C. 



□ 



Lemma 4. Assume the conditions of Theorem 1 ZioW. T/ien i/iere exist 
r± > 0, 5 > and e > snc/i £/ia£, /or aZZ z satisfying < ^(z) < ri and 
< <5, and /or a// a G T, X a (z) ^ 1, and /or all z satisfying IZ(z) = r\ 
and \1(z)\ < 5, and for all a G T, 

(4.9) \X a {z)-l\>e. 

Proof. Let := E w (£f) k . By integration by parts and Proposition 1, 

\ a (z) = l + nfz+ [ Z tX a{2) (z - t) dt 



at least for all z such that TZ(z) < r\ and |2r(z)l < 5, where ^ denotes the sec- 
ond derivative. Therefore, for all z in the set S := {z:lZ(z) < r±/2, \I(z)\ < 
o", \z\ < where Mi*)^ are defined in Lemma 3, and all a G T, we 

have 



\X a (z) - 1| > \fiz\ 



tX a{2) (z - t) dt 
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Take e > such that the square S £ := {TZ(z) < e, \1(z)\ < e] is contained in 
the set S; for example, e = ri/2 A 6 A (v^l) wm do. Then 

(4.10) \X a (z) -l\>^\z\ forallzGS e . 

By the assumption that {S% : a G T} is uniformly strong nonlattice and 
Proposition 1, we have, for all \0\ < 5, \X a (i9)\ = \E%e ie ^ \ + O{\0\)\ this 
implies that \\ a {i6) - 1| > g(e) > 0, for all \Q\ > e and all a G T. Take 
r := g(e)(2uj i ) A e > 0. Then, for all < u < r < ri, 6 > \9\ > e and a G T, 

|A°(u + ^) - 1| 

> \X a (i9) - 1| - |A a (n + id) - X a (i9)\ 

ru+ie 



>9{e) 



X a{2) {z)dz 



> g{e) - uv{ 



> g(e) - ^-v*^ = ^- L . 



Furthermore, (4.10) implies that \X a {u + i6) — 1| is positive for all < u < r, 
\6\ < e and a G T, and \X a (r + i6») - 1| is at least ^Jr/2 for all |0| < e. Thus, 
taking 5 := g(e)/2 A /2 > 0, the lemma is proved. □ 

Since the rate of convergence in the uniform Markov renewal theorem in 
Theorem 1 can be proved for each a G T, the uniformity in a G T appealing 
to Lemmas 3 and 4 when necessary, we will present the proofs by omitting 
a for simplicity. 

Let B = [s,s + h), and recall that U ( ~ a ' A \B) defined in (4.7) is the renewal 
measure. For simplicity, we delete a and denote 

oo 

(4.11) U^ A) (B) :=J2Pv{s<Sn<s + h,X n eA} 

n=0 

as the renewal measure of {(X n , S n ),n > 0}. 

To prove Theorem 1, we evaluate the Fourier transform of the renewal 
measure U A . As in Carlsson (1983) and Carlsson and Wainger (1982), we 
perform Fourier inversion of the Fourier transform as a generalized function. 
We refer the reader to Gelfand and Shilov (1964), Schwartz (1966) and 
Strichartz (1994) for the basic theory; in particular, the following notation 
and concepts will be used. 

A test function (f(s) is an infinitely differentiable function that vanishes 
outside a bounded region in R. Let V denote the linear space of all test 
functions, and V the space of linear functionals on P. A sequence (p n G T> 
is said to converge to zero if tp n and all its derivatives converge to uni- 
formly and vanish outside a common bounded subset of R. A generalized 
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function is a continuous linear functional on D. A function / denned on 
R for which / f(s)(p(s)ds is absolutely convergent for any ip G T> is called 
locally integrable. A C°° function / on R is of class T if / and all its partial 
derivatives are rapidly decreasing in the sense that they are of order 0(|s| _a ) 
as |s| — > oo, for every a > 0. Linear functionals on T are called tempered dis- 
tributions, and T denotes the set of all tempered distributions. The Fourier 
transform <p of a function ip G D is defined by ^>(0) = / (p(s) exp(i6s) ds for 
fl£R. The Fourier transform of a generalized function / is the linear func- 
tional / defined on the space {ip:ip is the Fourier transform of some ip G T>} 
by (2tt) = (/,£) for all (p &T>. 

As in renewal theory for simple random walks, the proof of Theorem 
1 requires detailed analysis of the characteristic function for the additive 
component S n . The analysis can be decomposed in two parts: for \9\ near 
zero and for \9\ away from zero. The rate of convergence for the renewal 
measure to the Lebesgue measure scaled by the mean is given by the analysis 
of \6\ near zero. The contribution of \9\ away from zero is negligible via 
the property of local integrability. That is, we need to show that for fleR, 
there exists a 5 > with \9\ > 5, J2nLo E 7T (e l6Sn ) and its fcth derivatives with 

respect to 9 are locally integrable for k = 1,2, By using kth. integration 

by parts, we thus need to show the following lemma. 

Lemma 5. Suppose {(X n , S n ), n > 0} is a strongly nonlattice Markov 
random walk satisfying (2.4), C1-C4 and C6. Then for every c > 0, for any 
r > 2 and k = 0, 1, . . . ,r, 

oo 

(4.12) sup^l^e^-^^oo. 

W>C n= Q 

Proof. For k = 0, . . . ,r and for any x G X, we have 

\E x (S k n e^)\<Y^\mn'--^ eS ")\, 

where the summation extends over jx,.-.,jk G {L ■•■ , n }- There are n k terms. 
We shall give upper bounds for each term. Fix ji,---,j^ G {1, . . . , n}, and 
a natural number m to be determined later. Let 

J = {j G {1, . . . ,n} : \j -jp\> 3m,p= l,...,k}. 

Divide J into blocks A\, B\, . . . , Ai, B[ as follows: define jx, . . . ,ji by 

ji = inf J and j p+1 = inf { j > j p + 7m : j G J} 

and let Z be the smallest integer for which the inf is undefined. Write 

A p = l[{e l6n - 1/2 ^ :\j-j p \<m}, p = 1, . . . , I, 
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b p = l[{e ien ~ 1/2 ^ :j P + m + l<j<j p+1 -m-l}, p = 1, 1, 
B l = U{e ien - 1/ ^:j>j l +m + l}, 

Then 



^■■■^oe ies -=l[A p B p R. 



We have 



I i 
E X R [] A p B p - E X R H B p E{A p \i 3 : j / j p ) 
l l 



<E 

9=1 



q-1 



E X R J] A pJ B p (A 5 - E(A p \Zj : j / j 9 )) 



(4.13) 



xl[B p E{A p \^:j^j p ) 

g+i 



i 

<E 

9=1 



9-1 



l 

i 

x JJ B p E{A p \^ : < |j -j p \ < 3m) 

9+1 

The first summation term in (4.13) vanishes since 

9-1 I 
Rl[A p B p and Y[B p E(A p \^:0<\j-j p \<3m) 

1 9+1 

are both measurable with respect to the cr-field generated by £j :j ^ j q . 
Recall that the functions 

E(A P \^ :0<\j-j p \<3m) for p = 1, . . . , I 

are weakly dependent since j p +\ — j p > 7m, p = I, ... ,1 — 1. Using condi- 
tion CI we obtain 



E x RY[B p E(A p \Cj :0<\j- j p \ < 3m) 



< (2nP) r E a 



jE(A p \^:0<\j-j p \<3m) 
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< {2n^) r Y[E x \E{A p \^ : < \j - j p \ < 3m) | 
l 

+ {2n p ) r l ■ 4d" 1 e- Q!m . 

With the strong nonlattice condition (2.3), the conditional strong nonlattice 
condition (2.4) and Lemma 2 in Statulevicius (1969), we find an upper bound 
for 

E x \E{A p \^:0<\j-j p \<3m)\. 

We have for \9\ > 5, the relation E x \E(A p \£j :j / j q )\ < e~ 5 , and hence 
by (2.4), for all 0€R,|0| <5, 

E x \E(A p \^:j^j q )\<exp(-5\9\ 2 /n). 

Therefore, for all 9 G R, 

E x \E{A p \^:0<\j -j p \<3m)\ 

< E x \E(A p \^ : j / j q )\ 

<max(exp(-<5|#| 2 /n),e~ 5 ). 

If we choose K appropriately and let m be the integral part of Klogn, 
then the assertion of the lemma follows from 

8\9\ 2 /n) n ' m < exv(-5\9\ 2 /(Klogn)) 

< exp(-5'n £ / 2 ) 

for \9\ > erf and some 5' > 0. □ 

Proof of Theorem 1. By using the same argument as that in The- 
orem 2.4 of Puh and Lai (2001), we have (2.5) and (2.6). The details are 
omitted. 

To prove (2.7), let g(s) S C°° and have support in Q = {s : |s| < 1}. Let 
g £ (s) = £g(s/£), let I a be the indicator function of the set A and let Q c = 
{s :\s\ < c} for c > 0. Let L be the measure with density ds/ri. As in Carlsson 
and Wainger (1982), we have that 

g e *I ai _ e *(U^-L){8)-Ke 

(4.14) < (U {A) -L)(Q + s) 

<g £ *In 1+E *(U^ -L)(s) + Ke, 

where K can be chosen uniformly for c bounded. Letting e = s~ k with k 
large enough, we have (2.7) if we can show that there exists r > such that 

(4.15) \g e * In c * (U^ - L)(s)\ < Ke~ rs . 
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To prove (4.15), we consider the Fourier transform 

oo 

i>{e) = Y,E v {z i6Sn hx{x n )} 

n=0 

(4.16) 

oo oo 

= J2 \ n (e)v,Q e hi + ]T v*P n 6 {I - Qe)h u 

n=0 n=0 

where Pg, Qg are defined in (2.11) with z = i9, and h\ := I a- Note that the 
second equation in (4.16) follows from Proposition l(i). 

As in Carlsson and Wainger (1982), there exists a 5 > such that, for 
\9\<6, 

(4.17) i;(9) = — - + ~6(e)\u*Q e hi + £ v*P n g{I - Q )hi, 

\ / / n=0 

where <5(-) denotes the Dirac delta function. By using Fourier inversion of 
the generalized function, we have 

(4.18) =>) + ^/ e^^i^tf 

l r 00 

(4.19) + — / e~ ies g E {&)IaMT, v * P e{I-^e)h 1 de 

(4.20) +— / e- i6s g £ (e)InMY,E 1/ {e idSn h 1 (X 1 )}de, 

where g e denotes the Fourier transform of g £ and Iq c denotes the Fourier 
transform of Iq c . 

Equation (4.18) can be analyzed by making use of the Taylor expansion 
of X(6) as that in Proposition 1, which says that for any r > 2 as \9\ — ► 0, 

1 \(*> 
(A(0)-1) =0(1) farfc<r-l, 



\0\ 

1 \W /l 

Tsr(W-i 



w 

(A(0)-l + i^)J =0(1) for£:<r-2, 

1 \(*> 
( A (0)-l + i^)J = ( 



-fc-2\ 



for k = r — 1 , r, 
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where denotes the kth. derivative. Also v*Qehi and its kth derivatives 
converge to as \9\ — > 0. 

Next, we want to verify that the rate of convergence in (4.18) is 0(e~ rs ) 
for some r > as s — > oo. Note that 

(4.21) J- f K{e- i6s g £ (0)i Q Mv*Qehi(i/v0)}d9 = ^ + O(e- rs ), 
2>n Jo<\e\<s 2fl 

as s — ► oo. Consider 

^ / e- tf 'g g (g)J ne ((?)i/ < .Q g / tl ( * - d9 

2t J0<\6\<6 \l-\(ld) fid J 

= J- lim/ / e- ie ^ £ (^)^ c (^)^Q^i 

(4.22) x ( i-— - -L) d6 



l-\(%6) 
+ / e- i0s g e (6)in c (O)v*Qohi 

Je<B<5 



-Ade 



l-\(i0) (j.6 

Let < u\ £ 0, where G is defined in K6. For any u £ (0, iti), and z = 
u + i8 £ C, consider four lines L\(e) = {z : 1Z(z) = 0,e < \1(z)\ <6}, L<i = 
{z:lZ{z) £ [0,u],X(z) =5}, L 3 = {z:7l(z) £ [0,u], J(z) = -5}, L 4 = {z:TZ(z) 
u,\X(z)\ < 8}, and one semicircle, L^(e), from e to — e, oriented clockwise. 
Define 

(4.23) h(z) = e- zs g e {z)I Qc {z)v*Qzhi f 



1 — X(z) nz, 

Since h is analytic in the regions from L\(e) to Ls(e), by Cauchy's theorem 
for contour integral, we have 

(4.24) 



h(z)dz+ / h(z)dz+ / h(z)dz 
Li(e) JL 2 JL 3 



+ / h{z)dz+ I h(z)dz = 0. 
Jl 4 Jl 5 ( £ ) 

The continuity h yields that the residual of h(z) at is 0, whence 

lim / h(z) dz = 0. 
Combining with the Riemann-Lebesgue lemma, we have 



lim / h(z) dz 

£ -> /Li(e) 
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-r 

(4.25) 



f e~( u+i9)s g £ (u + iO)i Qc (u + iO^Qu+ieh! 



1 



d.9 



l — X(u + i9) /x(u + i6 

= 0(e~ rs ) for some r > as s — > oo. 

To analyze (4.20), we make use of Lemma 5 which implies that J2ri=o E I/ {e lSSn } 
and its partial derivatives up to order r are bounded for any r > 2, and 
a fortiori locally integrable, in the region {6: \9\ > 5}. Moreover, Iq c and its 
derivatives are bounded by a constant times nf=i 1^1 _1 as 1^1 ~~ > 00 ■ There- 
fore N integrations by parts as in page 359 of Carlsson and Wainger (1982) 
can be used to show that (4.20) = 0{\ loge| /s^ ) for any N. Since rj(8) and 
its partial derivatives of order r are bounded for < \9\ <5* for any r > 2, 
N integrations by parts can also be used to show that (4.19) = 0(s^ N ) for 
any N. Therefore, by making use (4.18)-(4.20), (4.23) and (4.25), we have 
(4.15) and hence get the proof. □ 



5. Proof of Theorem 2. In this section, we assume the Markov random 
walk {(X n , S n ),n > 0} defined as (1.1) is uniformly ergodic with respect to 
a given norm, and the stationary mean = 0. Under the minorization con- 
dition CI, making use of the results in Hipp (1985), Malinovskii (1987) and 
Jensen (1989), we have the following asymptotic expansions of the density 
for the distribution in Markov random walks. 



Lemma 6. Assume C1-C5 with r = 3 in C3. We assume, without loss 
of generality, the asymptotic variance a 2 = 1 . Then 

(5.1) Pu{s n < v^i = *M + ^w^S + (i + isfr^t^-), 

where o(-) is uniform in s. Here <£(•) denotes the standard normal distribu- 
tion, (/)(■) denotes the standard normal density, andQi(s) = k/6(1 — s 2 ) + k u , 
where k = E^f + 3 £ t =i E^t+i + 3 E*=i ^6^+1 + 6E~t 2 =i ^£i& 1+ i6 1+i2 +i 
and k u = Et^i E v £f Note that k v = if v = ir. 

Furthermore, if P n {S n < s^/n} has a density p n ^ n (sy/n) , then 

(5.2) |v )n (sVn)^ = 0(*)(l + g^(a 3 -3a)) + (l + \s\ 3 )- 1 o 

where o(-) is uniform in s. 

In the following, we shall assume C1-C5 and C7 hold. Lemma 7 is taken 
from Theorem 5 of Fuh and Lai (1998); we include it here for completeness. 
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Lemma 7. Let r > 1. Assume sup x E x (^i) r+1 < oo, where £,f denotes 
the positive part of £i. Furthermore, assume there exists e > suc/i i/mi 
infa-P^I^! < — e|Xi = x} > 0. Then, E^S^ < oo and {S t q^ — 6,6 > 0} zs 
uniformly integrable under the probability P n . 

A Markov random walk is called lattice with span d > if d is the maximal 
number for which there exists a measurable function 7: X — > [0, 00), called 
the shift function, such that P{£i — j(x) + "f(y) £{■■■, —2d, —d, 0, d, 2d, . . -}\Xo = 
x,X\ = y} = 1 for almost all x,y G X. If no such d exists, it is called 
nonlattice. 

Let W(t), < t < 00, denote Brownian motion with drift /U and put 
t~w = T~w(b) = inf{t : W{t) > 6}. Define the inverse Gaussian distribution Git; [i, b) = P^{rw(b) < t} 
and H+(s) = (E^S^)- 1 / S P. + {S T+ > t} dt. 

Lemma 8. Assume P is nonlattice. Suppose b — > 00 and m — > 00 so that, 
for some fixed < £ < 00, 6 = Qm 1 ! 2 . Then for all <t,s < 00, 

P^r(6) < mt, 5 T(6) - 6 < s} — ► G(t; 0, C)#+(s)- 

Proof. Note that 

P 7r {r(6) > mt, SV(6) - 6 < s} 

= S 7r (P 7r {5 T ( f) ) - 6 < s\r(b) > mt,S m };T(b) > mt). 

Let 6 = 6 — 6 1//4 . Then by the central limit theorem for Markov random walks 
[cf. Theorem 17.2.2 of Meyn and Tweedie (1993)], we have 

E 7T (P n {S T ^ - 6 < s\r(b) > mt,S m };T(b) >mt,b< S m < 6) 

< P„{b <S m <b}^0. 

Moreover, it is also known that P n+ {r + < 00} = 1 and by Lemma 7, E n+ S T+ < 
00, and 

PA S r(b) ~ b < s \ T ( b ) > mt, S m < 6} 

x P n {X m G dx\r(b) > mt, S m G dv} 

= Jo lx Px ^ S < h - v ) ~( b ~ v )^ s}P n {X m G dx\r{b) > mt, S m G dv}. 

Since P n {r(b) > mt,S m < 6} > as 6 — > 00, therefore for v uniformly 
in {r(6) > mt,S m < 6}, P x {5' r ( 6 _ t) ) - (6 - v) < s} -> H+(s) as 6 -> 00 by 
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the Markov renewal theorem in Theorem 1. Hence, uniformly in {r(6) > 
mt, Sm < b}, as b — > oo, 

^{^(6) - &< s l r ( & ) > mt,5 m } ->• H + (s). 

Under irreducible assumption and the minorization condition CI, for r = 
2, assumption C3 ensures that the Poisson equation (4.1) has a solution A 
which satisfies E 7T (A(Xi)) 2 < oo. Therefore, by the invariance principle for 
Harris recurrent Markov chain [cf. Theorem 17.4.4 of Meyn and Tweedie 
(1993)], the limiting marginal distribution of r(b)/m is G(t;0,(). Hence, 

P 7r {r(6) > mt, S t q,) -b<s} 

= E 7T (P 7T {S T ^ b ) - b<s\r{b) > mt, S m };r{b) > mt, S m < b) + o(l) 

= H + (s)PAr{b) > mt, S m <b} + o(l) - - G?(t;0,C)). □ 

Lemma 9. Ze£ < ( < oo, and R m = S T (b) — Qm 1 / 2 . Then for any e > 0, 
P n {R m > em 1 / 2 } = oim" 1 / 2 ). 

Proof. By Markov's inequality we have 

P n {Rm > em l / 2 }=e~ l m~ 1 / 2 [ RmdP n 

which is o(m~ l l 2 ) by Lemma 7. □ 

Lemma 10. Let < C < °o, < s < oo, and m\ = m(l — (logm)~ 2 ). 
Then asm-> oo, 

(5.3) P w {mi < r < m,S m < (C - s)m 1/2 } = o(m" 1/2 ), 

(5.4) P^mi < r < m, S m > (( + s)m 1/2 } = o(m" 1/2 ). 

Proof. By Lemma 9, 

Pjmi < r < m, S" m > (C + s)m 1/2 } 

(5.5) = P^mi <T<m,Rm< \sm l/2 , S m >(( + s)m l/2 } + o(m" 1/2 ) 
< sup P n {S m - n > ±sm 1/2 } + o{m~ 1 ' 2 ). 

mi <n<m 

It is easy to see (5.5) is o^" 1 / 2 ). By using a similar argument, we have that 
(5.3) is o{m- 1 / 2 ). □ 

Proof of Theorem 2. Since we will consider time delay in the proof, 
we denote Sq = sq for convenience. Let P^ m ' s ° (A) = P w {A\Sq = so,S m = 
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s}. Set so — ra x ' 2 Ao and s = m 1 / 2 ^ for some Ao, Co < C- Let s' = 2b — s = 
m 1//2 (2C — Co) denote s reflected about b. Prom (5.2) in Lemma 6 and the 
Markov renewal theorem in Theorem 1, in which T has only one element, 
it follows as in Lemmas 7-10 that, for mi = m{l — (logm) -2 ) and some 

£m * 0, 

(5.6) P^ S ){ T < m} - pK s °' s ){ T <m u S T -b< m 1/2 e m ] = o(m~ 1/2 ) 
and 

(5.7) Pl m ' S0 ' s '*>{T <m}- p( m - s »- s V <m u S T -b< m l l 2 e m } = c^m" 1 / 2 ). 

Set A rn = {t < mi,S T — b < m 1 ' 2 e m }, and let L^ m \n,S n ) denote the like- 
lihood ratio of £i,...,£ n under p( m ' s °' s ^ relative to p( m ' s °' s \ p or a \\ n < 
m — no, 

(r n\ t(™)< Q \ — P~K,m-n{s — S n )p 7T)1n (s — go) 

[ ' \ n ^n)-- , , _ „ v- 7— -r, 

/^7r, m—nV' 5 l -'nJI J n,m\ s ^0/ 

where p,r,m(s) is defined in (5.2). By (5.6) and Wald's likelihood ratio identity 
for Markov chains 



(5.9) pK s o, S ) {r <m} = E (m,s ,s>) {L (rn) ^ ^ ^ } + q( 



m 



-i/2i 



Substitution of (5.8) into (5.9) and expansion with the aid of (5.2) gives the 
first-order result, 

p(m,so,s) {T <m} ^ exp [_2( C - AoXC " Co)]- 

This motivates the following reformulation of (5.9), which is justified by 
(5.7) and the fact that p( m ' s °' s '*> <{ T = m} = c^m" 1 / 2 ): 



p{m,s ,s) {T <m} _ exp [_ 2(C _ Ao )( C _ Co) ] + o( 



in 



-l/2i 



(5.10) 

= Bf 80 ' s) {i W (T, Sr) - exp[-2(C - A )(C - Co)]; A m }. 

The likelihood ratio of £i, • • - ,£n under p(_ m ' s °' s ) relative to -P^-f-ISo = *o} 
is p W)m -. n (s' — S n )/p n:m (s' — so). Hence by (5.8) and Wald's likelihood ratio 
identity once again, the right-hand side of (5.10) becomes 



En 

(5.11) 



Pir,rn— t\& 5V) 



- exp[-2(C - A )(C - Co)] Pn ' m - T ( iS f '~ S : ) ;A m S = a \. 



The rest of the proof of (2.9) involves use of Lemma 6 to expand the 
integrand in (5.11) and application of Lemma 8 to evaluate the resulting 
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expectation. Let R m = S T — m 1//2 £. Some tedious algebra gives that the 
integrand in (5.11) equals 

[(l-r/m^VCCo-Ao)]- 1 

y { J C-Co + Rm/m 1 ' 2 } _ f C-Co-Rm/m 1 / 2 - 
X \*\ (l- r /m)V2 J 9 { (l-r/m)V2 

which can be expanded to give 

-2(1 - r/m)- 1 / 2 exp[i(Co - A ) 2 - ±(C - Co) 2 /U - r/m)] 

(5.12) 

x [(C - Cv)R m /m l l 2 {\ - r/m)] + o((l + i^)/m) 

uniformly on A m . According to Lemma 8, r/m and i? m are asymptotically 
independent, converge in law, and by Lemma 7, R m is uniformly integrable. 
Also, (5.12) is a bounded, continuous function of r/m on A m . Hence, (5.12) 
can be substituted into (5.11) and Lemma 8 applied to evaluate the result. 
Putting sq = and performing the appropriate integrations yields (2.9). 
Formally, (2.10) follows by substituting (2.9) into 

(5.13) P.{r <m,S m <c}= f P^M{ T < m}P 7V {S m G ds}. 

J (— oo,c) 

However, some care is required to justify this calculation, especially in the 
case c = b (7 = £), when s in (5.13) can be arbitrarily close to b. It is easy 
to see that (2.9) holds uniformly on each compact subinterval of (— oo,£); 
but if Co - ► C; (5-12) is not necessarily bounded, (5.6) may fail to hold, and 
indeed the proof of (2.9) disintegrates. 

To circumvent this difficulty, we need to apply the duality argument 
to a time-reversed Markov chain. By condition C7, recall that P X (A) = 
J A p{x,y)M(dy) for all A E A, where p(x,-) = dP x /dM. Letting Q x ' y {B) = 
P(£i G B\Xo = x, X\ = y), we can express the transition probability function 
(1.1) as 

(5.14) P(x,AxB)= f p(x,y)Q x >y(B)M(dy):= f F(x,y; B)M(dy), 

J A J A 

where F(x,y;B) = p(x , y)Q x,y (B) . For ease of notation, we still denote tt as 
the density of the invariant probability measure. We shall use ~ to refer to 
the time-reversed (or dual) chain {(X n , S n ),n > 0} with transition kernel: 

(5.15) F(y, x; B) = F(x, y; B)n{x)/^{y). 

Let r* = sup{n :n < m, S m > b}, and let f = inf{n : S n > b} denote the 
first passage time of the time-reversed Markov random walk S n at the linear 
boundary b. Observe that Pi m ' '°{r <m} = P^ { T * > 0} = P^ fi) {f < 
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m}; so to approximate Pi m,0 '^{T < m} for £ — s < Co < C> ^ suffices to con- 
sider P^ m ' A,< '^{f < m} for £ = and ( — e < Ao < C (recall that Ao = m _1 / 2 so). 
It is easy to see from (5.10)-(5.12) that uniformly for £ — e < Ao < so — 

m" 1 / 2 < C, J^ m ' so ' 0) {f < m} = exp{-2C(C - A )} + o{m~ 1 / 2 ), which suffices 
to justify formal substitution of (2.9) into (5.13) for £ in a neighborhood of 
b and complete the proof when 7 = C- D 

6. Proof of Theorem 3. The way to prove Theorem 3 is a suitable ap- 
plication of Theorem 1 via the following lemmas. 

Lemma 11. Assume the conditions of Theorem 3 hold. Then, there exists 
5 > and \a\ < 5 such that the induced Markov chain {{X", S%),n > 0} 
with transition probability (2.13) is aperiodic and irreducible. Moreover, it is 
uniform ergodic with respect to a given norm and satisfying K1-K6. 

PROOF. For \a\ < 5, it is known [cf. Ney and Nummelin (1987)] that 
{(X%, S%),n > 0} is an aperiodic and irreducible Markov chain. S ince {(X n , S n 
n > 0} satisfies CI, P a is geometrically e~ A ( a ) -recurrent for \a\ < 5, and 
therefore is e -A ( a )-uniformly ergodic, compare Theorem 4.1 of Ney and 
Nummelin (1987). 

Define 

T „,, V(dy x ds)e- K ^ +as r(y-a) 
* a (dy x ds) = f— — yi-J-, 

where (\&r)(a) is a normalizing constant, and 

h a {x) = {^r)(a)r~ 1 {x]a)h{x), 
where \l/(-) and h(-) are defined in CI with 

P(x,dy x ds) > h(x)fy(dy x ds). 

This implies that 



P a (x,dy x ds) > 



^?(dy x ds)e 



-h(a)+as 



r(y;a) 



(\Pr)(a:)r 1 (x;a)h(x) 



(*r)(a) 

= h a (x)^ a (dy x ds). 

Therefore the mixing condition Kl hold. 

To prove the moment condition K6, denote X a (z) as the eigenvalue of P '. 
By (2.13), we have 

\E«{e e ^)-E^)\ 

rj^a)_ 0s / r^a^ _ e as-A(a)\ dy x ds ^f dx ) 
x, y <=x J-00 r(x;a) \r(y;a) J 



< 



' x,ydX J —00 



r(y;a) 



r(x; a) 



r{x;a) _ QS _ A ( a ) 



r{y;a) 



P a (x,dy x ds)n a (dx) 
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for 9 £ C R. From lim a |o \ r {y] a)/r(x; a)\ = 1, we get lim a |o su P6»l-^v ~~ 
E"(e e '^ 1 )| = by dominated convergence. Therefore, we may choose a* > 
so that sup ae r 0)Q ,*i supg l-E^e — E 7T (e e ^ 1 )\ < C, say. Hence, condition C6 
implies K6 holds. By using a similar argument, we also have the moment 
conditions K2-K4. □ 



Lemma 12. Assume the conditions of Theorem 3 hold. Then there exists 
a* > such that the family {(X% , S®),n > 0:0 < a < a*} satisfies (2.3) and 
(2.4). 

Proof. Since the proofs of (2.3) and (2.4) are similar, we only prove 
(2.3), the uniformly strong nonlattice case. Let \ a (z) denote the eigenvalue 
of P 1 . Since { (X n , S n ),n>0} is assumed to be strongly nonlattice, we have 
that g(l) := inf| fl |>i |1 - E n (e ie ^)\ > 0. However, by (2.13), 



E v (e i6il ) - E*(e ie S? 



x,yeX 



< 



x,y€X J —oo 



r(y,a) | 
r(x; a) 

r(y;a) 



i6s I as—A(a) 



r(x; a) 



-(x; a) 



r(y;a) 
r(x; a 



P a (x,dy x ds)7r a (dx) 



r(y;a) 



P a (x,dy x ds)ir a (dx) 



for all real 9. Since lim^o \r(y;a)/r(x;a)\ = 1, so lim Q | su Pe I ^ (e 1 ^ 1 ) — 
E%(e ie t? )| = by dominated convergence theorem. Therefore, we may choose 
a* >0sothatsup ae[0ia , ] sup e | J B 7r (e ie « 1 )-^(e^?)| <#(l)/2, say. Choosing 
a* in this way, by applying the definition of g(l) and the triangle inequality, 
we obtain 



inf inf \1-E"(e ie ^)\ > g(l)/2. 
ae[0,a*] \0\>1 1 71 v t\-u\ >l 



Consider 



9(S) :- 



inf inf |1 

a£[0,a*] \6\>8 



we want to show that g(5) > for all S > 0. If S > 1, then g(5) > g(l)/2 > 0. 
Suppose < 5 < 1. To show g(5) > 0, it suffices to show 

inf inf \l-EaJe ie ^)\ > 0. 

aG[0,a*]<5<|6>|<l 

However, since E%{e i9 Z?) = E%{e( a+i9 ^)/ E%{e a &), it is easy to see that 
E^(e ld ^) is a continuous function of (a, 6) for a € T and real 9. Therefore, 
|1 — E®(e *i )|, being a continuous function on the compact set {(a, 9) :0 < 
a < a*, 5 < |#| < 1}, must attain its minimum there. To complete the proof, 
we need only show that this minimum value cannot be 0. Supposing to the 



MARKOV RENEWAL THEORY 



35 



contrary that E^{e l9 ^) = 1 for some < a < a* and 5 < \9\ < 1, we would 
have 

is an integer j = 1. 

However, by the assumption that {(X n , S n ),n > 0} is strongly nonlattice 
and the property of exponential embedding that P n is absolutely continuous 
with respect to P", this is a contradiction. □ 

Use the same notation as the paragraph before Theorem 2 in Section 2. 
Note that r n ,, r + and r_ depend on a; we omit it here for simplicity. The 
following lemma is related to uniform strong nonlattice of the ladder chains. 
The proof is a straightforward generalization of Theorem 6 in Fuh and Lai 
(1998) and is omitted. 

Lemma 13. Assume the conditions of Theorem 3 hold. Let P® + be the 
transition probability of the ladder Markov chain { (X^ , S" ) , n > 0}. Then, 
there exists a* > such that, for < a < a* , the family { ( X" n , S" n ) , n > 0} 
is uniformly strong nonlattice. 

The following lemma generalizes Lemmas 4.4 and 4.5 in Heyde (1964) for 
simple random walks. 

Lemma 14. Assume the conditions of Theorem 3 hold. Then, there exist 
a* > 0, r\ > and C such that, for all a £ [0, a*], 

E«{e riS ?+)<C. 

Proof. Under the assumption C6 and Lemma 11, for a £ T C R, we 
can define the linear operators P a , P, and Q on M as in (2.11). By the 
spectral decomposition theory for linear operator on the space M developed 
in Proposition 1, we have for h £ J\f, 

(6.1) E v {e aS "h(X n )} = \ n (a)^Q a h + ^P™(/ - Q a )h, 

where Q Q is defined in (2.12). It also can be shown that there exist K* > 
and < 5* < 5 such that, for |q| < 5* , 

(6.2) |KP2(/ - Q a )h\\ < K*\\h\\\a\{(l + 2p)/3} n , 

and under assumption C6, it follows from Proposition 1 that A (a) has the 
Taylor expansion 

r 

(6.3) A(a) = l + ^AjV/j! + A(a) 
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in some neighborhood of the origin, where A (a) = 0(|a| r ) as a — ► 0. 
Now, for such a, we have for all — oo < s < oo, 

Pu{S n <s)< e- as (\ n (a)v*Q a l + v*V n a {I - Q Q )1), 

where 1 denotes the identity function. Also, under the assumptions fi < 
and C6, there exists sufficiently small a such that A(a) < 1. Along this with 
(6.2), there exists C > such that e c X(a) < 1 and for all c, < c < C, 

oo 

(6.4) jT i e cn P v (S n <s) <oo. 

n=l 

Next, for all 7 G (0,1), define F (s) = I{ s >o}, Fi(a) = P„(Si < s) and 
F n (s) = P !/ (5 n < s;maxi< fc < n _i S'fc < log 7), for n > 1. Then, (6.4) implies 
that 

00 

(6.5) 5>"\F n (log 7 )<cx), 

71=1 

for some r > 0. 

Note that the probability p n of the first passage time T^y) out of the 
interval (log 7, 00) for the Markovian random walk S n is n is given by 

(6.6) p n = F n _i(log7) - F n (log7), n>\. 

By (6.5) and (6.6), we have E u e tr ^ < 00 for some t > 0, for all 7 G (0, 1). 
Hence 



(6.7) ^ 7r (e t?1 )<oo implies E n (e tSr +) < 



00. 



Using the requirement in the definition of the exponential embedding that 
r must contain an interval about 0, take any positive a\ G T. Let C := 
F7r(e Ql5r +); by (6.7), Cis finite. Since < ini\ a \ >SyX£X r(x;a) < sup| a | >5ia . eA -r(a;;a) < 
00, then, if we take a* and r\ both to be cx\/2, say, for any a G [0, a*] we 
have 

nS°± x _K f r(X T+ ;a) j ri+a)s T+ ^(a) 



I r(X ;a) 
< E jr_(X I±2 a) e{ri+a)ST 
I r(X ;a) 

<E n {e aiST +} = C, 

which is the desired property. □ 

Lemma 15. Assume the conditions of Theorem 3 hold. Suppose b — > 00 
and < a [ such that, for some —00 < 1? < 00, ab — > Then for < t, s < 
00: 
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(i) PfMb) < b\ S? {b) -b<s}^ G(t;0, l)H+(a) and 

(n) Kim {b) - by, r{b) < 00} -> t^S-. 



Proof, (i) Let m = b 2 t and Tn be the <7-algebra generated by {(X n ,S n ), 
n < N}. By Wald's likelihood ratio identity for Markov chains, we have that 
for any stopping time N, a',a",A& J-n, and for each fixed x G X, 



(6.8) 



P«{An(N<oo)} 



r(XjV '^ exp((c/ - a")S N - N(A(a') - A(a"))) dP, 
An(7V<oo) r(x;a'j 



And this implies that 

P«{t <m,S«-b<s} 
~r(X T ; a 



(6.9) 



expjaSV — rA(a)}; r <m,S T — b< s 



■ exp(ab)E w 



r(X T ;a) 
r(X ;a) 



exp{a(5 r — b) — rA(a)}; 



t <m, S T — b < s 



It follows that as < a j 0, ab — > r(X T ;a)/r(Xo;a) — ► 1 and A (a) 



7^0? ~ |$ 2 /fe 2 . Hence at least for all finite s, Lemma 7 shows that the right- 
hand side of (6.8) converges to 

exp^)E n [exp{-^ 2 T W (l)};T W (l)<t]H + (s) 

= E v [exp{W(Tw(X)) ~ l$ 2 T W (l)};T W (l)<t]H + (s) 

= pW{r w (l)<t}H + (s) 

= G{t;$,l)H+(s). 

That this calculation is also valid when s = 00 follows from the Markov 
renewal Theorem 1 once it is known that E n {e rSr +) < 00, for some r > 0. 
This holds by Lemma 14. 

(ii) The proof of the convergence of E n+ S n+ follows from Lemma 6. The 
rest is a similar calculation and is omitted. □ 



By using the exponential martingale (2.13), uniform renewal theorem in 
Theorem 1 and Lemmas 11-15, the proof of Theorem 3 is similar to that of 
Theorem 2 and is omitted. 
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APPENDIX 

Characteristic functions of uniform Markov random walks. Here we gen- 
eralize the work of Fuh and Lai (2001). We include it for completeness. Using 
the same notation and assumptions as in the first paragraph after Theorem 
2 of Section 2, define P ™, P, z/f and Q on M as (2.11). Condition K2 ensures 
that P" and P are bounded linear operators on jV, and (2.2) implies that 

(A.l) ||P n -Q||= sup ||P n /i-Q/i|| < 7 p n . 

hGAf: \\h\\ = l 

For a bounded linear operator T:Af ^Af, the resolvent set is defined as 
{y E C : (T — yl) -1 exists} and (T — yl)~ l is called the resolvent (when the 
inverse exists). From (A.l) it follows that, for y ^ 1 and \y\ > p, 

oo 

(A.2) R(y) := Q/(y - 1) + £ (P» - Q)/^ +1 

n=0 

is well defined. Since R(y)(P — yl) = —I = (P — yI)R(y), the resolvent 
of P is —R(y). Moreover, by K3 and an argument similar to the proof of 
Lemma 2.2 of Jensen (1987), there exist K > and r/ > such that, for 
\z\ < ??, \y-l\>(l- p)/6 and \y\ > p + (1 - p)/6, ||P° - P|| < #|a| and 
R%(y) E^o-WKP? ~ P)-R(y)}" is well defined. Since 7^(y)(P^ - 
yl) = R a z {y){{V a z - P) + (P - yl)} = -7= (P« - y7)7^(y), the resolvent 
of P? is -i?«(y). 

For \z\ < rj, the spectrum (which is the complement of the resolvent set) 
of P" therefore lies inside the two circles C\ = {y : \y — 1| = (1 — p)/3} and 
C*2 = {y : 1 2/ 1 = p + (1 — p)/3}. Hence by the spectral decomposition theorem 
[cf. Riesz and Sz-Nagy (1955), page 421], Af = Afi(z) B Af 2 {z) and 

(A.3) Q« := ^ ^ 2£(y) dy, 7 - Q« := ^ ^ 2£(y) dy 

are parallel projections of Af onto the subspaces Afi(z), Af 2 (z), respectively. 
Moreover, by an argument similar to the proof of Lemma 2.3 of Jensen 
(1987), there exists < 5 < rj such that Bi(z) is one-dimensional for \z\ < 8 
and sup| 2 | <( 5 ||Q° — Q|| < 1. For \z\ < 5, let X a (z) be the eigenvalue of P" 
with corresponding eigenspace Af\(z). Since is the parallel projection 
onto the subspace Bi(z) in the direction of 62(2), (2.12) holds. Therefore, 
for heAf, 

E u {e zS "h(X n )} = v?P ™h = v?P « n {Qz Qz)}h 
= (\ a (z)fv«Q a z h + „?P «»(/ - Q£)/i. 

Suppose K4 also holds. An argument similar to the proof of Lemma 2.4 
of Jensen (1987) shows that, there exist < 5* < 6 and K* > such that for 
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\z\ <S*, |i^P^ n (/-Q^)/i| <K*\\h\\ w \z\{{l + 2p)/3} n . Moreover, analogous 
to Lemmas 2.5-2.7 of Jensen (1987), it can be shown that X a (z), z^"Q"/i 
and ^Qi/"P" n (/ — Qz)h have continuous partial derivatives of order [r] 
for \z\ < 5* . Furthermore, we have the following proposition. 

Proposition 1. Assume the conditions of Theorem 1 hold. Let h€.N 
and there exists a 5 > such that z € C and \z\ <S. 

(i) E u {e zS "h(X n )} = {\ a (z)) n v«Q«h + v«(P°) n (I - Q™)h. Moreover, 
there exist < 8* < S, 0<7<1 and K > such that, for \z\ < 5*, X a (z), 
z^"Q"/i and X)n^=o v *(^"z ) n (I ~ Q")' 1 have continuous partial derivatives of 
order [r], and 

\v«(P°) n (I- qg)h\ </^||/i|||z|7 n for all n > 1. 

Furthermore, 

A Q (0) = 1, VA Q (0) =iTfi a , V 2 A a (0) = -TV a T'. 

(ii) Define f%{z) =J2™ =0 E l/ (e zS Zl {Xn€A} ), and let h A {z) = l {x€A} . Then 
for < \z\ < 5*, 

f%{z) = (1 - a q (^))- 1 ^q^ j4 + v a (z), 

where r] a (z) has continuous partial derivatives of order [r] and ij a (z) = 
0(\z\) as 2->0. 
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